Holistic Grading

The Colorado Supreme Court Board of Law Examiners employs a holistic grading method to score Multistate Essay and Performance Exams.

Prior to implementing holistic grading in February 2013, exam answers were scored in three phases: First Scoring, Regrade, and Reconciliation. With the implementation of holistic grading, the Regrade and Reconciliation phases were eliminated. This decision was not made for purposes of convenience or expediency. Rather, it was intended to improve the uniformity, reliability, and integrity of the scoring process by having all answers graded at one time when the graders are focused and calibrated.

Colorado employs holistic grading for the MEE and MPT. Holistic grading assigns a score to an answer based on a global impression of the answer’s quality. Although grading guidelines outline specific elements that should be included in an answer to the question, the grade assigned to a paper goes beyond a simple tally of the elements covered and assigns a global score that includes the overall quality of the answer. All answers are scored using the same 1-6 score scale. The 1-6 scale is a relative scale, that is, a “6” is better than a “5,” which is better than a “4,” etc. The National Conference of Bar Examiners uses this 1-6 scale to train graders.

As a Uniform Bar Exam jurisdiction, all examination results are final and not open to regrading or appeal. It is not possible to regrade one essay answer without having to regrade the entire exam pool.

Grading Process

The Colorado grading teams and members of the Colorado Law Committee participate in a national calibration session with other UBE jurisdictions, as well as a second team-specific calibration session just before grading commences to ensure that members on a grading team remain calibrated during the entire grading process. In the grading process for Colorado, grading teams must start and end the grading for all MEE and MPTs during the same designated time. Each grading team member is given a grading packet that does not contain any applicants’ identifying information. Within each grading packet there are embedded answers at regular intervals that are the same for each member of a grading team throughout the entire packet. When a grader reaches an embedded answer, each team member will grade the same essay answer, make notes, and assign their suggested score. To ensure calibration, the suggested score for that embedded answer should be the same for all graders. When there is a variation, the team must discuss why they awarded the essay that particular score, then re-review their previous essays up to the last embedded answer to ensure consistency. This process takes several days; this is the only activity that graders engage in during that timeframe; this is not an activity they fit in to their workday or do at night. It is their sole job for several days. Colorado’s grading procedures provide for all graders to remain focused, calibrated, and in constant communication with their grading partners during the entire grading time, ensuring fairness and consistent application of grading guidelines.

Scoring

Scoring-MBE: Raw MBE scores (the number of questions correctly answered) are converted to scaled scores by the National Conference of Bar Examiners using a statistical formula designed to make scores comparable from exam to exam.

Scoring-MEE/MPT: Essay and performance test question raw scores are converted to scaled scores by the National Conference of Bar Examiners using a statistical formula designed to make scores comparable. The MEE and MPT scores are scaled to the MBE, with the MBE weighted 50%, the MEE 30%, and the MPT 20%. That score is then added to the MBE scale score to arrive at a total UBE score.

All raw essay scores are proofed by several individuals before the scores are sent to NCBE for scaling to the MBE; scaled scores are then proofed by at least two individuals after they are uploaded to applicants’ records in CiviCore.